Information processing apparatus, information processing system, and program

ABSTRACT

[Object] To provide an information processing apparatus by which sound can be smoothly re-listened to. 
     [Solution] There is provided an information processing apparatus including: a reproduction processing unit that performs reproduction of a recorded sound on a basis of a reproduction start instruction for starting re-listening of the recorded sound from a position tracking back a predetermined time from a reproduction start time, at which the reproduction start instruction is input, to a position of a present time.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims the benefit under 35U.S.C. § 120 of U.S. patent application Ser. No. 15/767,525, titled“INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING SYSTEM, ANDPROGRAM,” filed Apr. 11, 2018, which is a U.S. National Stage Entryunder 35 U.S.C. § 371 of International Application No.PCT/JP2016/074784, filed in the Japan Patent Office on Aug. 25, 2016,which claims priority to Patent Application No. 2015-205389, filed inthe Japan Patent Office on Oct. 19, 2015, each of which is incorporatedby reference herein in its entirety. U.S. patent application Ser. No.15/767,525 issued as U.S. Pat. No. 10,666,995 on May 26, 2020.

TECHNICAL FIELD

The present disclosure relates to an information processing apparatus,an information processing system, and a program.

BACKGROUND ART

A function of re-listening of missing sound when a user misses somesound in a hearing-aid apparatus is provided. According to PatentLiterature 1, for example, a past sound signal to be listened to againand current input sound are provided at the same time after processingis performed thereon such that the past sound signal and the currentinput sound can be easily distinguished.

CITATION LIST Patent Literature

-   Patent Literature 1: JP 2012-134919A

Non-Patent Literature

-   Non-Patent Literature 1: Morita and Itakura, “Expansion and    compression of sound in time axis using pointer interval controlled    overlap and add (PICOLA) and evaluation thereof”, S61. 10, pp.    149-150-   Non-Patent Literature 2: Noguchi, Bannai, Hata, and Kataoka    “Distinction and removal of sudden noise in 1-channel input signal”,    Journal of The Acoustical Society of Japan, pp. 655 to 656, March    2004-   Non-Patent Literature 3: H. Kinoshita, M. Delcroix, T. Nakatani    and M. Miyoshi, “Suppression of late reverberation effect on speech    signal using long-term multiple-step linear prediction”, IEEE Trans.    Audio, Speech and Lang. Proc., Vol. 17, No. 4, pp. 534-545, 2009

DISCLOSURE OF INVENTION Technical Problem

However, there are many cases in which users that use hearing-aidapparatuses generally have difficulty in listening to sounds of aplurality of speakers. Therefore, even if two sounds can bedistinguished as in Patent Literature 1 described above, there is apossibility that the simultaneous sound provision is not a sufficientfunction as re-listening reproduction. In addition, since the sounditself to which users desire to listen again is sound to which the usershas once missed listening, it is necessary for the users to listen tothe sound with further concentration when the users hear the soundagain. From this point of view, it is not possible to state thatsimultaneous provision of past sound and current sound can provide soundto which users can easily listen, and the simultaneous provision is notsufficient as the re-listening reproduction function.

In addition, there is also a hearing-aid apparatus provided with a voicememo function capable of performing sound recording on sound necessaryfor users. However, since it is necessary for the users to press abutton or the like to explicitly provide instructions for starting andcompleting the sound recording and an instruction for reproduction, thevoice memo function is inconvenient for promptly reproducing there-listening sound during conversation.

Thus, the present disclosure proposes a novel and improved informationprocessing apparatus, information processing system, and program bywhich sound can be smoothly re-listened to.

Solution to Problem

According to the present disclosure, there is provided an informationprocessing apparatus including: a reproduction processing unit thatperforms reproduction of a recorded sound on a basis of a reproductionstart instruction for starting re-listening of the recorded sound from aposition tracking back a predetermined time from a reproduction starttime, at which the reproduction start instruction is input, to aposition of a present time.

In addition, according to the present disclosure, there is provided aninformation processing system including: a first device that acquiresand outputs sound; and a second device that includes a reproductionprocessing unit that performs reproduction of a recorded sound acquiredby the first device on a basis of a reproduction start instruction forstarting re-listening of the recorded sound from a position trackingback a predetermined time from a reproduction start time, at which thereproduction start instruction is input, to a position of a presenttime.

Further, according to the present disclosure, there is provided aprogram for causing a computer to function as: a reproduction processingunit that performs reproduction of a recorded sound on a basis of areproduction start instruction for starting re-listening of the recordedsound from a position tracking back a predetermined time from areproduction start time, at which the reproduction start instruction isinput, to a position of a present time.

Advantageous Effects of Invention

Thus, the present disclosure proposes a novel and improved informationprocessing apparatus, an information processing system, and a program bywhich re-listening sound can be smoothly reproduced. Note that theeffects described above are not necessarily limitative. With or in theplace of the above effects, there may be achieved any one of the effectsdescribed in this specification or other effects that may be graspedfrom this specification.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a functional block diagram illustrating a minimum functionalconfiguration of a hearing-aid system according to a first embodiment ofthe present disclosure.

FIG. 2 is a functional block diagram illustrating another functionalconfiguration of the hearing-aid system according to the embodiment.

FIG. 3 is an explanatory diagram illustrating a re-listeningreproduction screen as a display example of an information presentationunit according to the embodiment.

FIG. 4 is a flowchart illustrating an example of re-listeningreproduction processing performed by the hearing-aid system according tothe embodiment.

FIG. 5 is an explanatory diagram illustrating a reproduction state inStep S100 in FIG. 4.

FIG. 6 is an explanatory diagram illustrating reproduction states inSteps S120 and S130 in FIG. 4.

FIG. 7 is an explanatory diagram illustrating a reproduction state inStep S150 in FIG. 4.

FIG. 8 is an explanatory diagram illustrating a reproduction state inStep S180 in FIG. 4.

FIG. 9 is a flowchart illustrating another example of the re-listeningreproduction processing performed by the hearing-aid system according tothe embodiment.

FIG. 10 is a functional block diagram illustrating a minimum functionalconfiguration of a hearing-aid system according to a second embodimentof the present disclosure.

FIG. 11 is a functional block diagram illustrating another functionalconfiguration of the hearing-aid system according to the embodiment.

FIG. 12 is a hardware configuration diagram illustrating a hardwareconfiguration of an information processing apparatus according to theembodiment.

MODE(S) FOR CARRYING OUT THE INVENTION

Hereinafter, (a) preferred embodiment(s) of the present disclosure willbe described in detail with reference to the appended drawings. Notethat, in this specification and the appended drawings, structuralelements that have substantially the same function and structure aredenoted with the same reference numerals, and repeated explanation ofthese structural elements is omitted.

Hereinafter, a hearing-aid system that provides a function by whichsound can be re-listened to in real time will be described as apreferred mode of an information processing system according to thepresent disclosure. Such a hearing-aid system performs reproduction of arecorded sound on a basis of a reproduction start instruction forstarting re-listening of the recorded sound from a position trackingback a predetermined time from a re-listening start time, at which are-listening start instruction is input, to a position of a presenttime. If the reproduction of the recorded sound reaches the presenttime, only ordinary real-time sound is provided. A user can smoothlyperform re-listening of desired sound merely by performing are-listening start operation. In addition, the following descriptionwill be given on the assumption that a sound monitor includes a hearingassistant and a hearing-aid that is a medical tool.

In addition, the description will be given in the following order.

1. First embodiment (separate type: hearing-aid system)

1.1 System configuration

(1) Minimum configuration

(2) Applied configuration

1.2 Re-listening reproduction processing

(1) Case in which only instruction for starting re-listeningreproduction is provided

(2) Case in which there is instruction for completing re-listeningreproduction

1.3 Supplementary note

(1) Non-stationary noise suppression processing

(2) Simultaneous hearing of re-listening sound and real-time sound

1.4 Conclusion

2. Second embodiment (integrated type: hearing-aid apparatus)

(1) Minimum configuration

(2) Applied configuration

3. Hardware configuration examples

1. First Embodiment 1.1 System Configuration

First, a functional configuration of a hearing-aid system according to afirst embodiment of the present disclosure will be described withreference to FIGS. 1 and 2. The hearing-aid system according to theembodiment includes an audio device 100 that acquires sound and outputssound and an information processing device 200 that performs control forsound reproduction by the audio device 100.

(1) Minimum Configuration

First, a minimum functional configuration of the hearing-aid systemaccording to the embodiment will be described on the basis of FIG. 1.The hearing-aid system illustrated in FIG. 1 enables execution of are-listening reproduction function in addition to an ordinaryhearing-aid function as minimum functions.

(Audio Device)

The audio device 100 is a component that is worn by a user on his/herears and used, and for example, is an earphone, a headphone, or the likethat is used by being connected to a device provided with an informationprocessing function, for example. Although a closed type of wearing onboth ears is assumed as a shape of the earphone, the present disclosureis not limited to such an example. The audio device 100 includes a soundinput unit 110, an audio device data transmission unit 120, an audiodevice data reception unit 130, a signal synthesis selection unit 140,and a sound output unit 150 as illustrated in FIG. 1.

The sound input unit 110 is a sound monitor unit that acquires externalsound, and includes, for example, one or a plurality of microphones, anA/D converter, and the like. The sound input unit 110 converts asurrounding audio signal into digital waveform data and outputs thedigital waveform data to the audio device data transmission unit 120.

The audio device data transmission unit 120 transmits the digitalsignal, which has been input from the sound input unit 110, to theinformation processing device 200. The digital signal may be transmittedthrough wired transmission or wireless transmission. In a case of thewireless transmission, for example, the transmission can be implementedby Bluetooth (registered trademark) or the like.

The audio device data reception unit 130 receives encoded waveform dataof re-listening sound which has been received from the informationprocessing device 200. The audio device data reception unit 130 decodesreceived streaming data into raw waveform data (PCM) on which signalprocessing can be performed and outputs the data to the signal synthesisselection unit 140.

The signal synthesis selection unit 140 performs processing ofgenerating a sound signal to be output from the sound output unit 150 inaccordance with an input from a control input unit 220, which will bedescribed later, and outputs the sound signal to the sound output unit150. For example, the past sound received from the informationprocessing device 200 is output when re-listening reproduction of theacquired sound is performed, and the re-listening sound is not output inthe other cases.

The sound output unit 150 is an output unit that provides sound to auser and includes, for example, a D/A converter, a driver unit, or thelike. The sound output unit 150 converts sound digital data afterprocessing is performed thereon by the signal synthesis selection unit140 into an audio signal and outputs the audio signal.

In the audio device, the sound input unit 110 and the sound output unit150 may perform independent operations or non-independent operations foreach of left and right ears in a case of being wearable on both theears.

(Information Processing Device)

The information processing device 200 is a device provided with aninformation processing function, and for example, is an informationprocessing terminal or the like, such as a smartphone, capable ofpairing with the audio device 100, for example. The informationprocessing device 200 includes a re-listening reproduction processingunit 210, a control input unit 220, a first sound storage unit 231, anda second sound storage unit 233.

The re-listening reproduction processing unit 210 is a functional unitthat performs processing for the re-listening reproduction of therecorded sound, and includes an information processing device datareception unit 211, a chasing reproduction signal processing unit 213,and an information processing device data transmission unit 215 asillustrated in FIG. 1.

The information processing device data reception unit 211 receives adata stream from the audio device data transmission unit 120, decodesencoded data, and acquires sound waveform data. The informationprocessing device data reception unit 211 outputs the acquired soundwaveform data to the chasing reproduction signal processing unit 213.

The chasing reproduction signal processing unit 213 analyzes the sounddata input from the information processing device data reception unit211 and performs processing of outputting sound from a time trackingback a predetermined time in accordance with the input from the controlinput unit 220. Note that, hereinafter, a sound signal that reaches auser's ears after the signal processing is performed thereon by thechasing reproduction signal processing unit 213 will also be referred toas “re-listening sound”, while a sound signal that reaches a user's earswithout not being subjected to the signal processing by the chasingreproduction signal processing unit 213 will also be referred to as“real-time sound”. The chasing reproduction signal processing unit 213can reproduce the re-listening sound with shortening a reproduction timein accordance with the input from the control input unit 220. Forexample, at least a part of the reproduction sound may be reproduced ata faster reproduction speed than the unmultiplied speed. The processingmay, for example, apply a technology of performing reproduction withincreasing a speech speed without changing a sound pitch thereof (seeNon-Patent Literature 1, for example). The chasing reproduction signalprocessing unit 213 outputs the generated reproduction signal to theinformation processing device data transmission unit 215.

In addition, as another time-shortening reproduction method, a timesection in which no human voice is present or a no-sound section may beautomatically detected as a skip target section, and reproduction may beperformed while skipping at least a part of the skip target section. Forthe detection of the skip target section, determination may be made onthe basis that a time section during which a sound volume level does notexceed a predetermined threshold value continues for a predeterminedtime or longer is skipped, for example. In addition, it is also possibleto simultaneously perform the time-shortening by changing thereproduction speed and the time-shortening by the skip reproduction.

The information processing device data transmission unit 215 encodes thesound after the processing is performed thereon by the chasingreproduction signal processing unit 213 and transmits the encoded soundto the audio device 100.

The control input unit 220 controls the functions of the audio device100 and the information processing device 200 in response to operationinputs from a user. For example, the control input unit 220 causesexecution of the re-listening reproduction function and causescompletion of the re-listening reproduction function on the basis ofoperation inputs from a user. In addition, the control input unit 220performs various kinds of setting of a sound volume and the like of theaudio device 100 on the basis of an operation input from a user. Inaddition, a device that is directly operated by a user is not limited tothe information processing device 200 although not illustrated in thedrawing. For example, it is also assumed that a user input is providedto the control input unit 220 by the user operating a computer, a remotecontroller, or the like that is a separate entity from the informationprocessing device 200 to establish communication between the computer,the remote controller, or the like and the information processing device200

The first sound storage unit 231 stores sound that has been received bythe information processing device data reception unit 211 and has beenacquired by the audio device 100. The sound recorded by the first soundstorage unit 231 can be used as recorded sound for the re-listeningreproduction and the like. Information such as the sound recorded in thefirst sound storage unit 231 may be recorded in the second sound storageunit 233 in parallel therewith. In this case, the second sound storageunit 233 may be used for the re-listening reproduction and the likeinstead of the first sound storage unit 231. For example, the firstsound storage unit 231 may be a volatile memory while the second soundstorage unit 233 may be a nonvolatile memory or a hard disk drive.

In addition, all information related to recorded sound, a sound featureamount, or time stamp recording in the first sound storage unit 231 orthe second sound storage unit 233 may be accumulated and recorded aslong as the capacity thereof permits, or only the latest informationcorresponding to a predetermined time length may be recorded. Since theformer recording method accumulates information corresponding to alonger time, it is possible to enhance a degree of freedom in chasingreproduction. According to the latter recording method, it is possibleto save the capacity of the storage unit used and not to unnecessarilyhold information in consideration that the recorded information isinformation related to privacy of other persons.

(2) Applied Configuration

Next, a configuration example of a hearing-aid system that is achievedby further adding functions to the hearing-aid system with the minimumconfiguration illustrated in FIG. 1 as a base will be described on thebasis of FIG. 2. The hearing-aid system illustrated in FIG. 2 includes asound analysis function, a noise cancellation function, and the like inaddition to the hearing-aid function and the re-listening reproductionfunction that are minimum functions. Hereinafter, the functions newlyadded to the hearing-aid system with the first-time configurationillustrated in FIG. 1 will be mainly described, and description of thesame functional configuration as that of the hearing-aid system in FIG.1 will be omitted.

(Audio Device)

As illustrated in FIG. 2, the audio device 100 includes the sound inputunit 110, the audio device data transmission unit 120, the audio devicedata reception unit 130, and the sound output unit 150 in the similarmanner to that in the audio device 100 illustrated in FIG. 1. Further,the audio device 100 in FIG. 2 includes a sound feature calculation unit121, a time stamp information generation unit 123, a first signalsynthesis selection unit 141, a real-time sound processing unit 143, asecond signal synthesis selection unit 145, and a cancelation signalgeneration unit 160.

The sound input unit 110 is a sound monitor unit that acquires externalsound and is configured in a similar manner to that in the sound inputunit 110 in FIG. 1. The sound input unit 110 converts a surroundingaudio signal into digital waveform data and outputs the digital waveformdata to the sound feature calculation unit 121, the first signalsynthesis selection unit 141, and the cancelation signal generation unit160. In addition, the converted digital signal is also output to theaudio device data transmission unit 120 in response to an instructionfrom the control input unit 220.

The sound feature calculation unit 121 calculates a sound feature amountnecessary for operations of the chasing reproduction signal processingunit 213 or the sound analysis processing unit 212 of the informationprocessing device 200 from the waveform data input from the sound inputunit 110. The sound feature calculation unit 121 calculates, as thesound feature amount, an amplitude ratio and a phase difference perspecific time between the respective microphones, for example. The soundfeature amount calculated by the sound feature calculation unit 121 isoutput to the audio device data transmission unit 120.

The time stamp information generation unit 123 generates time stampinformation to be applied to data that is transmitted to the informationprocessing device 200 by the audio device data transmission unit 120 inresponse to an input from the control input unit 220. The time stampinformation generation unit 123 generates the time stamp information tobe applied to a time section of a digital signal to be transmitted, forexample, in a case in which there is an operation of not transmitting adigital signal of a no-sound time section to the information processingdevice 200, or the like. The time stamp information generated by thetime stamp information generation unit 123 is transmitted to the audiodevice data transmission unit 120. This is a method for solving aproblem that information about which time a continually sent signalcorresponds to is lost and it is not possible to perform re-listening bytracking back a predetermined time in the information processing device200 in a case in which the operation of not transmitting the digitalsignal of the no-sound time section to the information processing device200 is performed or the like as described above.

The audio device data transmission unit 120 outputs a digital signalinput from the sound input unit 110 in a similar manner to that inFIG. 1. The digital signal may be transmitted through wired transmissionor wireless transmission. In a case of the wireless transmission, forexample, the transmission can be implemented by Bluetooth (registeredtrademark) or the like. In addition, the audio device data transmissionunit 120 can also transmit the output from the real-time soundprocessing unit 143, which will be described later, instead of the soundsignal input from the sound input unit 110 and can also dynamicallychange the output signal. In addition, the audio device datatransmission unit 120 according to the embodiment can also compressinformation of the sound feature amount calculated by the sound featurecalculation unit 121 and transmit the information along with the soundwaveform data to the information processing device 200. At this time,the audio device data transmission unit 120 performs simple bit depthconversion and then transmits the sound feature amount. In addition, ina case in which the sound data is respectively acquired by a pluralityof microphones, the audio device data transmission unit 120 may extractonly an input from a predetermined microphone, perform encodingprocessing thereon by sub-band codec (SBC), encode, decode, and thentransmit the information.

For example, the audio device data transmission unit 120 may sample asignal derived from an input of one microphone of a sound monitor forthe left ear with 8 kHz, pack the sound waveform data encoded by SBC andthe sound feature amount data after simple bit depth conversion for eachconversion length frame and transmit the packed data. At this time, thetransmission data may be transmitted by a serial port profile (SPP) onthe basis of a defined unique protocol. In addition, the encoding anddecoding scheme can be arbitrarily selected, and an advanced audiodistribution profile (A2DP), a headset profile (HSP), a hands-freeprofile (HFP), or the like may be provided in the system as in FIG. 1that does not use the sound feature amount, for example. The data may betransmitted by the unique protocol encoding scheme using SPP, A2DP, HSP,or HFP.

The sound feature calculation unit 121 functions to securefunctionalities of the sound analysis processing unit 212 and thechasing reproduction signal processing unit 213 while reducing theamount of data transmitted and received between the audio device 100 andthe information processing device 200. For example, since irreversiblecoding is used for transmitting and receiving data, and a channel and aband for sending the data are limited in the description in the previousparagraphs, the data transmitted from the audio device data transmissionunit 120 to the information processing device data reception unit 211 isthe original sound waveform data from which a part of information hasbeen lost. Therefore, processing that can be implemented by the soundanalysis processing unit 212 and the chasing reproduction signalprocessing unit 213 using this waveform data is limited. For example,such an event that it is not possible to calculate the feature amountrequiring input sound waveforms of both the ears and the feature amountusing high-frequency information lost due to the band limitation and thelike occur. Thus, in a case in which there is a feature amount that hasto be calculated by using the original waveform data in feature amountsnecessary for the processing on the side of the information processingdevice 200, it is possible to implement a desired function by the soundfeature calculation unit 121 calculating the feature amount andtransmitting the feature amount to the information processing device200.

Further, the audio device data transmission unit 120 can also choose notto transmit the data in a case in which no human voice is included incurrent input sound, for example, on the basis of sound feature amountdata calculated by the sound feature calculation unit 121. In this case,time stamp information of the time section of the transmitted soundwaveform data is transmitted to the information processing device datareception unit 211 along with the sound waveform data transmitted fromthe audio device data transmission unit 120 to the informationprocessing device data reception unit 211. The sound feature calculationunit 121 in this case functions to further reduce the amount of datatransmitted and received between the audio device 100 and theinformation processing device 200 unlike the description in the previousparagraphs.

The audio device data reception unit 130 receives the encodedre-listening sound streaming data received from the informationprocessing device 200. The audio device data reception unit 130 decodesthe received streaming data to waveform data (such as PCM) on whichsignal processing can be performed and outputs the waveform data to thefirst signal synthesis selection unit 141 or the second signal synthesisselection unit 145. Alternatively, it is also possible to dynamicallyselect which of the first signal synthesis selection unit 141 and thesecond signal synthesis selection unit 145 the input is to be provided,in accordance with the input from the control input unit 200. In a casein which the audio device 100 and the information processing device 20are configured to be separate from each other as in the hearing-aidsystem according to the embodiment, antennas or the like for theencoding, the decoding, and the wireless transmission are needed, and aconfiguration in which a part of these antennas are included in the samedevice can be employed. In this case, exchange of data between blocks inthe same device can be implemented by simple data exchange using ashared memory without compressing and expanding data.

The first signal synthesis selection unit 141 and the second signalsynthesis selection unit 145 perform processing of generating the soundsignal output from the sound output unit 150 in response to the input ofthe control input unit 220 and output the sound signal to the soundoutput unit 150. The input signal from the audio device data receptionunit 130 to only any one of the first signal synthesis selection unit141 or the second signal synthesis selection unit 145 is effective, andthe input signal to the other is output without any change.

Specifically, the first signal synthesis selection unit 141 and thesecond signal synthesis selection unit 145 perform processing onreal-time sound and re-listening sound, respectively, in response to theinput from the control input unit 220, synthesize the sound, and outputthe synthesized sound to the sound output unit 150. In a case in whichonly the real-time sound is reproduced, for example, an output signal isobtained by multiplying the real-time sound by “1,” multiplying there-listening sound by “0,” and taking the sum thereof. In addition, in acase in which only the re-listening sound is reproduced, the outputsignal is obtained by multiplying the real-time sound by “0,”multiplying the re-listening sound by “1,” and taking the sum thereof.Further, the output signal may be obtained by performing separateprocessing on the real-time sound and the re-listening sound andsuperimposing both the real-time sound and the re-listening sound.

The real-time sound processing unit 143 is a sound processing unit thatperforms signal processing with low delay on the sound obtained from thesound input unit 110 in the similar manner to the general sound monitor.The real-time sound processing unit 143 executes, for example, beamforming processing by multiple microphones, feedback cancellation orsuppression processing, stationary/non-stationary noise suppression,sound amplification, equalizing, processing performed by a compressor orthe like, and a user can listen to the output in real time.

The sound output unit 150 is an output unit that provides sound to auser and includes, for example, a D/A converter, a driver unit, or thelike. The sound output unit 150 converts sound digital data, on whichprocessing has been performed by the first signal synthesis selectionunit 141 and the second signal synthesis selection unit 145, into anaudio signal and outputs the audio signal.

The cancelation signal generation unit 160 performs signal processing toimplement a noise cancellation function. The cancelation signalgeneration unit 160 generates a cancelation signal that cancels soundthat leaks from the outside and reaches eardrums even if the soundmonitor is mounted. The generated cancellation signal is synthesizedwith the output from the real-time sound processing unit 143 and is thenoutput from the sound output unit 150.

In addition, the sound input unit 110, the real-time sound processingunit 143, and the sound output unit 150 in the audio device 100 mayperform independent operations for each of the left and right ears ormay perform non-independent operations.

(Information Processing Device)

The information processing device 200 is a device provided with aninformation processing function, and for example, is an informationprocessing terminal or the like, such as a smartphone, capable ofpairing with the audio device 100. The information processing device 200includes the re-listening reproduction processing unit 210, the controlinput unit 220, the first sound storage unit 231, and the second soundstorage unit 233 as illustrated in FIG. 2. Further, the informationprocessing device 200 according to the embodiment includes aninformation presentation unit 240.

The re-listening reproduction processing unit 210 is a functional unitthat performs processing for reproduction of recorded re-listening soundand includes an information processing device data reception unit 211, asound analysis processing unit 212, a chasing reproduction signalprocessing unit 213, and an information processing device datatransmission unit 215 as illustrated in FIG. 2.

The information processing device data reception unit 211 receives adata stream from the audio device data transmission unit 120, decodesencoded data, and acquires sound waveform data. The informationprocessing device data reception unit 211 outputs the acquired soundwaveform data to the sound analysis processing unit 212 and the chasingreproduction signal processing unit 213.

The sound analysis processing unit 212 analyzes the sound feature amountand the sound waveform data input from the information processing devicedata reception unit 211 and extracts information necessary for chasingreproduction signal processing and presentation of the information to auser. As the analysis for the chasing reproduction signal processing,the sound analysis processing unit 212 may extract information bycalculating power of a signal and an autocorrelation coefficient in aspecific time from the waveform data, for example, and applying a lowpass filter. Then, the sound analysis processing unit 212 may regard atime as a “sound present section” when the values of the both exceedthreshold values in a predetermined time and output the information tothe chasing reproduction signal processing unit 213.

In addition, as analysis for the presentation of information, the soundanalysis processing unit 212 may extract the information by taking atemporary difference in the sound feature amount (an amplitude ratio anda phase difference between microphones) calculated by the sound featurecalculation unit 121, for example, and applying the low pass filter.Then, the sound analysis processing unit 212 may regard a position atwhich the value exceeds a certain threshold value as “a change point ofa human speaker” and output the information to the informationpresentation unit 240. In addition, if a case in which the informationprocessing device 200 is a smartphone and a sound waveform is displayedon a display that is an information presentation unit 240 is consideredas another specific example, processing for displaying the waveform withhigh visibility in a limited display area on the display can beperformed by the sound analysis processing unit 212. At this time, thesound analysis processing unit 212 may display, with vertical lines,ranges of the maximum value and the minimum value for several samples(for example, 005 sec), rather than displaying the sample value itself,after a band pass filter to pick up a waveform only in a sound band ofhumans from the sound waveform data, for example, is applied. In thismanner, it is possible to present the sound waveform data to a user in amanner in which the sound waveform data is easily understood.

The chasing reproduction signal processing unit 213 analyzes sound datainput from the information processing device data reception unit 211 andperforms processing to output sound from the time tracking back apredetermined time in accordance with an input from the control inputunit 220. In addition, the sound signal that reaches a user's ears aftersignal processing is performed thereon by the chasing reproductionsignal processing unit 213 will be referred to as “re-listening sound”below. Meanwhile, the sound signal that reaches a user's ears withoutthe signal processing performed by the chasing reproduction signalprocessing unit 213 will be referred to as “real-time sound”. Thechasing reproduction signal processing unit 213 can shorten the time ofthe re-listening sound and reproduce the re-listening sound in responseto the input from the control input unit 220. For example, at least apart of the re-listening sound may be reproduced at a fasterreproduction speed than the unmultiplied speed. The processing may, forexample, apply a technology of performing reproduction while increasinga speech speed without changing a sound pitch thereof (see Non-PatentLiterature 1, for example). The chasing reproduction signal processingunit 213 outputs the generated reproduction signal to the informationprocessing device data transmission unit 215.

In addition, as another time-shortened reproduction method, a timesection in which no human voice is present or no-sound section may beautomatically detected as a skip target section, and reproduction may beperformed while skipping at least a part of the skip target section. Forthe detection of the skip target section, determination may be made onthe basis that a time section during which a sound volume level does notexceed a predetermined threshold value continues for a predeterminedtime or longer is skipped, for example. In addition, it is also possibleto simultaneously perform the shortening by changing the reproductionspeed and the time-shortening by the skip reproduction.

The information processing device data transmission unit 215 encodes thesound on which the chasing reproduction signal processing unit 213 hasperformed processing and transmits the sound to the audio device 100.

The control input unit 220 controls the functions of the audio device100 and the information processing device 200 in response to operationinputs from a user. For example, the control input unit 220 causesexecution of the re-listening reproduction function or causes completionof the re-listening reproduction function on the basis of an operationinput from a user. In addition, the control input unit 220 performsvarious setting for a sound volume and the like of the audio device 100on the basis of an operation input from a user. In addition, the devicethat is directly operated by a user is not limited to the informationprocessing device 200 although not illustrated in the drawing. Forexample, it is also assumed that a user input is provided to the controlinput unit 220 by the user operating a computer, a remote controller, orthe like that is a separate entity from the information processingdevice 200 to establish communication between the computer, the remotecontroller, or the like and the information processing device 200

The first sound storage unit 231 stores the sound which has beenreceived by the information processing device data reception unit 211and has been acquired by the audio device 100. At this time, the soundfeature amount and the time stamp information may be stored together inthe first sound storage unit 231. The sound recorded in the first soundstorage unit 231 can be used as recorded sound in the re-listeningreproduction. The information such as the sound recorded in the firstsound storage unit 231 may be recorded in the second sound storage unit233 in parallel therewith. In this case, the second sound storage unit233 may be used in the re-listening reproduction or the like instead ofthe first sound storage unit 231. For example, the first sound storageunit 231 may be a volatile memory while the second sound storage unit233 may be a nonvolatile memory or a hard disk drive.

In addition, all information related to recorded sound, a sound featureamount, or time stamp recording in the first sound storage unit 231 orthe second sound storage unit 233 may be accumulated and recorded aslong as the capacity thereof permits, or only the latest informationcorresponding to a predetermined time length may be recorded. Since theformer recording method accumulates information corresponding to alonger time, it is possible to enhance a degree of freedom in chasingreproduction. According to the latter recording method, it is possibleto save the capacity of the storage unit used and not to unnecessarilyhold information in consideration that the recorded information isinformation related to privacy of other persons.

The information presentation unit 240 is a functional unit that displaysa still image, a moving image, letter information, and the like on thebasis of the information input from the sound analysis processing unit212 and, for example, is a display or the like. In addition, theinformation presentation unit 240 includes not only the display itselfbut also a display processing unit that causes the display to displayinformation. In addition, the information presentation unit 240 is notnecessarily integrated with the information processing device 200although not illustrated in the drawing. That is, the informationpresentation unit 240 may be incorporated in a device that is physicallyseparated from the information processing device 200, receive the stillimage, the moving image, the letter information, and the like to bepresented through a moving image, sound information, and the likethrough wired or wireless communication, and present the still image,the moving image, the letter information and the like to a user. Theinformation presentation unit 240 that is physically separated from theinformation processing device 200 as described above includes, forexample, a television, a tablet terminal, a PC, or the like.

Here, FIG. 3 illustrates a re-listening reproduction screen 10 as adisplay example of the information presentation unit 240 according tothe embodiment. The re-listening reproduction screen 10 is a screen forperforming various kinds of setting for the re-listening reproductionfunction. In addition, the re-listening reproduction screen 10 is notlimited to the configuration illustrated in FIG. 3 and can beappropriately changed.

A timeline display area 11 that displays, as an image, the soundwaveform data that has been obtained by buffering the data in a pastpredetermined time (for example, 60 seconds) and has been input from thesound analysis processing unit 212 may be displayed on the re-listeningreproduction screen 10. Lines (for example, lines 11 a to 11 d) thatsection a start position and a completion position of a speech sectionmay be displayed at positions corresponding to times of change points ofa speaker, which has been analyzed by the sound analysis processing unit212, for example. Further, a line L_(Tp) indicating a currentre-listening reproduction position may be displayed in a sound waveformdata image. For example, in a case in which the information presentationunit 240 includes a touch sensor or the like so as to be able todirectly operate information displayed in the information presentationunit 240 with a finger or the like, the re-listening reproductionposition can also be changed by changing the position of this lineL_(Tp) on the time axis.

Further, the re-listening reproduction screen 10 may present informationindicating a current operation status of the chasing reproduction signalprocessing unit 213. For example, how long before (for example, how manyseconds before) the present time the sound that is currently output bythe chasing reproduction signal processing unit 213 and is heard by auser through the audio device 100 is may be displayed. In addition, are-listening reproduction start button (Playback button) 12, there-listening reproduction completion button (Stop button) 13, and are-listening position setting unit 14 may be provided in there-listening reproduction screen 10 as illustrated in FIG. 3. It ispossible to start the re-listening reproduction processing if there-listening reproduction start button 12 is pressed, and it is possibleto complete the re-listening reproduction processing if the re-listeningreproduction completion button 13 is pressed. In addition, there-listening position setting unit 14 can set how long the time is to betracked back from the re-listening reproduction start time to reproducethe recorded sound. In the example illustrated in FIG. 3, the sound isreproduced from a time tracking back three seconds from the re-listeningreproduction start time.

In addition, the re-listening reproduction screen 10 can also include aspeech speed adjustment unit 15 that sets a reproduction speed in there-listening reproduction processing. In the example in FIG. 3, thereproduction speed in the re-listening reproduction processing can beincreased or decreased by sliding a knob portion 15 a. In addition, itis also possible to change the reproduction speed in a plurality ofdivided sections in the re-listening reproduction processing, and in thecase, speech speed adjustment units 15 may be provided to correspond tothe number of sections in which the reproduction speed can be set. Inaddition, the reproduction speed may be able to be set by inputting anumerical value, such as a “1.5-time speed”, for example, in addition tothe example in FIG. 3, and an icon, letters, or the like indicating thata skip mode is set may be displayed in a case in which the skip mode inwhich non-sound sections are skipped is set.

1.2. Re-Listening Reproduction Processing

Hereinafter, two cases will be described for the re-listeningreproduction processing by the hearing-aid system according to theembodiment. In the following two cases, the description will be given onthe assumption that the time tracking back from the re-listening startposition is set in advance such that “sound in the past five secondsfrom the present is re-listened to”, and a simultaneous hearing of thereal-time sound and the re-listening sound is not performed. Inaddition, the description will be given by using an example in whichreproduction is performed at a faster speed than the unmultiplied speedas a time-shortened reproduction method performed by the chasingreproduction signal processing unit 213.

(1) Case in which Only Instruction for Starting Re-ListeningReproduction is Provided

First, the re-listening reproduction processing performed by thehearing-aid system according to the embodiment will be described on thebasis of FIGS. 4 to 8. In addition, FIG. 4 is a flowchart illustratingthe re-listening reproduction processing performed by the hearing-aidsystem according to the embodiment. FIGS. 5 to 8 are explanatorydiagrams for explaining the re-listening reproduction processingperformed by the hearing-aid system according to the embodiment. Inaddition, Tnow represents a position at a present time, and Tprepresents a re-listening reproduction position in FIGS. 5 to 8. Inaddition, T₁ represents a time at which a re-listening reproductionoperation is input, T₂ represents a time tracking back tracking-backtime set from T₁ toward the past, and T₃ represents a time at which there-listening reproduction position reaches the present time.

The flowchart of the re-listening reproduction processing illustrated inFIG. 4 illustrates a case in which only an instruction for starting there-listening reproduction is provided. That is, the flowchartillustrates a case in which no operation is performed to move on to thereproduction of the real-time sound after there is an operation inputfor starting the re-listening reproduction.

In the re-listening reproduction processing performed by the hearing-aidsystem according to the embodiment, the control input unit 220 startsthe re-listening reproduction processing by the re-listeningreproduction processing unit 210 if a user inputs an operation forstarting the re-listening reproduction (S100) first, as illustrated inFIG. 4. The operation for starting the re-listening reproduction may bean operation of pressing the re-listening reproduction start button 12in the re-listening reproduction screen 10 illustrated in FIG. 3, forexample. The control input unit 220 causes the re-listening reproductionprocessing unit 210 to start the re-listening reproduction processingand provides an instruction to the audio device 100 such that soundoutput from the sound output unit 150 is only the re-listening sound(S110). Then, the reproduction position is changed to a positiontracking back a predetermined tracking-back time (S120), andreproduction of the re-listening sound after sound emphasis processingis started (S130). In addition, the sound emphasis processing is notnecessarily performed in Step S130, and the re-listening sound may bereproduced without performing the sound emphasis processing.

As illustrated in FIG. 5, for example, the operation of starting there-listening reproduction in Step S100 is assumed to be performed at atime after elapse of the time T₁ from the start of recording. Thereal-time sound is output to the user until the operation is performed,and the real-time sound is recorded in the first sound storage unit 231.After only the re-listening sound is output in Step S110 by the startingof the re-listening reproduction, the reproduction of the re-listeningsound is started from the time T₂ tracking back a predeterminedtracking-back time from the time T₁ as illustrated in FIG. 6 in StepS120. In the example in FIG. 6, the tracking-back time is set to fifteenseconds, and if the time T₁ at which the operation of starting there-listening reproduction is performed is fifteen minutes and 30seconds, for example, the re-listening sound is repeated from theposition of the time T₂ tracking back fifteen seconds therefrom (thatis, fifteen minutes and 15 seconds).

The re-listening sound is reproduced at a first reproduction speed inthe section of the reproduction of the re-listening sound in Step S130.The first reproduction speed may be set in advance or may be set by theuser. Since the user desires to check the content of the sound in thesection again, the first reproduction speed is generally set to be areproduction speed that is equal to or slower than the unmultipliedspeed. It is a matter of course that the first reproduction speed may beset to a reproduction speed that is faster than the unmultiplied speed.

Returning to the description of FIG. 4, the reproduction of there-listening sound in Step S130 is performed until the re-listeningreproduction position Tp reaches the re-listening start time T₁ (S140).Then, if the re-listening reproduction position Tp reaches there-listening start time T₁, the chasing reproduction signal processingunit 213 reproduces the re-listening sound at a second reproductionspeed in a section from the position Tp to the present time positionTnow (S150). Since it is necessary to cause the re-listeningreproduction position Tp to reach the present time in the section, thesecond reproduction speed is set to be a faster speed than theunmultiplied speed. Then, when there is no re-listening sound, which hasbeen buffered and has not yet been reproduced, in the first soundstorage unit 231 (S160), that is, when the re-listening reproductionposition Tp reaches the present time position Tnow, the re-listeningreproduction processing performed by the chasing reproduction signalprocessing unit 213 is completed. Then, the control input unit 220provides an instruction such that the sound output from the sound outputunit 150 to the audio device 100 is only the real-time sound (S170), andonly the real-time sound after the sound emphasis processing by thereal-time sound processing unit 143 is provided to the user (S180).

FIG. 7 illustrates a reproduction state of the re-listening sound afterthe re-listening reproduction position Tp reaches the re-listening starttime T₁. As illustrated in FIG. 7, the reproduction is performed at thesecond reproduction speed that is faster than the unmultiplied speed inthe section in Step S150. Thereafter, if the re-listening reproductionposition Tp reaches the present time position Tnow, the reproduction ofthe re-listening sound is completed, and only the real-time sound isoutput as illustrated in FIG. 8.

(2) Case in which there is Instruction for Completing Re-ListeningReproduction

Next, another case of the re-listening reproduction processing performedby the hearing-aid system according to the embodiment will be describedon the basis of FIG. 9. In addition, FIG. 9 is a flowchart illustratingthe re-listening reproduction processing in a case in which there is aninstruction for completing the re-listening reproduction. That is, acase is shown in which, after there is an operation input for startingthe re-listening reproduction, an operation input for completing there-listening reproduction is received from a user to move on to thetime-shortened reproduction of the re-listening sound and thereproduction of the real-time sound. Hereinafter, different processingfrom the re-listening reproduction processing illustrated in FIG. 4 willbe mainly described, and detailed description of the same processingwill be omitted.

In the re-listening reproduction processing performed by the hearing-aidsystem according to the embodiment, the control input unit 220 startsthe re-listening reproduction processing by the re-listeningreproduction processing unit 210 if an operation of starting there-listening reproduction is input from a user (S200) first, asillustrated in FIG. 9. The control input unit 220 starts there-listening reproduction processing by the re-listening reproductionprocessing unit 210 and provides an instruction to the audio device 100such that the sound output from the sound output unit 150 is only there-listening sound (S210). Then, the reproduction position is changed toa position tracking back a predetermined tracking-back time, and thereproduction of the re-listening sound after the sound emphasisprocessing is started (S220). Processing in Steps S200 to S220 is thesame as the processing in Step S100 to S130 in FIG. 4. In addition, thesound emphasis processing is not necessarily performed in Step S220, andthe re-listening sound may be reproduced without performing the soundemphasis processing.

Here, it is assumed that there is an operation input for completing there-listening reproduction from the user during the reproduction of there-listening sound in Step S220 until the re-listening reproductionposition Tp reaches the re-listening start time T₁ (S230). The chasingreproduction signal processing unit 213 moves the re-listeningreproduction position Tp to the re-listening start time T₁ in responseto the operation input (S240) and reproduces the re-listening sound atthe second reproduction speed in a section from the re-listening starttime T₁ until the re-listening reproduction position Tp reaches thepresent time position Tnow (S250). Since it is necessary to cause there-listening reproduction position Tp to reach the present time in thesimilar manner to that in Step S150 in FIG. 4 in this section, thesecond reproduction speed is set to be a faster speed than theunmultiplied speed.

Then, when there is no re-listening sound, which has been buffered andhas not yet been reproduced, in the first sound storage unit 231 (S260),that is, when the re-listening reproduction position Tp reaches thepresent time position Tnow, the re-listening reproduction processing bythe chasing reproduction signal processing unit 213 is completed. Then,the control input unit 220 provides an instruction to the audio device100 such that the sound output from the sound output unit 150 is onlythe real-time sound (S270), and only the real-time sound after the soundemphasis processing performed by the real-time sound processing unit 143is provided to the user. The processing in Step S250 to S280 is the sameas the processing in Step S150 to S180 in FIG. 4. As described above, itis possible to hear the real-time sound during the re-listeningreproduction processing in the section of Step S250 even in a case inwhich the user explicitly completes the re-listening reproductionprocessing.

Although the case in which there is an operation input for completingthe re-listening reproduction from the user until the re-listeningreproduction position Tp reaches the re-listening start time T₁ has beendescribed in the example in FIG. 9, the user can perform the operationinput for completing the re-listening reproduction even after there-listening reproduction position Tp reaches the re-listening starttime T₁. In this case, the chasing reproduction signal processing unit213 may further increase the reproduction speed of the re-listeningsound or may complete the reproduction of the re-listening sound itself,for example.

In addition, a case in which the user successively changes there-listening reproduction position Tp is also considered, and in thecase, the reproduction of the re-listening sound may be started bytracking back a predetermined time from a time at which the operationfor starting the re-listening reproduction is input every time the inputis made.

1.3. Supplementary Note

Since the hearing-aid system according to the embodiment can performre-listening reproduction of an acquired sound signal, it is possible toexecute sound processing of predicting the sound signal or processingthat takes a computation time. As a result, it is possible to achievesound emphasis with higher quality.

(1) Non-Stationary Noise Suppression Processing

For example, the hearing-aid system according to the embodiment canexecute non-stationary noise suppression processing. For example,non-stationary noise of collision of some objects such as keyboardtyping sound or door opening or closing is disagreeable to the ears andis a target to be suppressed. Meanwhile, since consonant parts of soundare important for hearing the sound, it is necessary to surely reproducethe parts. However, the consonant parts of sound and the non-stationarynoise such as the collision noise as described above have a commonfeature that energy is instantaneously focused on a high frequencyregion, and it is difficult to distinguish the both merely by observinginstantaneous signals.

Thus, it is possible to distinguish and remove sudden noise withsatisfactory accuracy by the chasing reproduction signal processing unit213 executing the non-stationary noise suppression processing tosuppress the non-stationary noise such as the collision sound on theassumption that the signal has been buffered and is to be reproducedwith a time delay to some extent. The signal processing technologydescribed in Non-Patent Literature 2, for example, can be applied to thenon-stationary noise suppression processing. Since the method describedin Non-Patent Literature 2 uses, as clues, signals before and after atime section to be processed, it is difficult to perform the method bythe real-time sound processing unit 143 due to a relatively largecomputation amount and necessary processing time. However, there-listening reproduction processing of the hearing-aid system accordingto the embodiment can perform the non-stationary noise suppressionprocessing since temporal restriction and computation cost are generous.

(2) Simultaneous Hearing of Re-Listening Sound and Real-Time Sound

In a case in which sound that is desired to be repeatedly heard is long,and a user himself/herself is required to respond in some way during thesound, it is preferable to provide a function of simultaneously hearingthe re-listening sound and the real-time sound. Thus, the hearing-aidsystem according to the embodiment can implement the simultaneoushearing of the re-listening sound and the real-time sound whileincluding the function of reproducing the re-listening sound.

The sound can be synthesized by the first signal synthesis selectionunit 141 and the second signal synthesis selection unit 145. However,there is a possibility that it becomes difficult to hear both there-listening sound and the real-time sound if both the sound is simplyadded to each other. Therefore, a sound signal of at least any one ofthe re-listening sound and the real-time sound may be processed suchthat the both can be easily discriminated from each other. Although thesound signal may be processed by the first signal synthesis selectionunit 141 and the second signal synthesis selection unit 145, the chasingreproduction signal processing unit 213, which has less restriction ofcomputation cost and less restriction of computation time, may processthe re-listening sound.

Specifically, it is possible to process the re-listening sound to hearlike a dry and vicinity sound as compared to the real-time sound byperforming echo suppression processing described in Non-PatentLiterature 3, for example. Alternatively, it is also possible to processthe re-listening sound as if the sound were generated in an environmentdifferent from that of the real-time sound, by applying a head transferfunction or a room impulse response (or something with equivalenteffect) filter. The processing is assumed to include measures oflocalization in the head, making a sound source position unclear, andthe like. Further, a pitch of the re-listening sound may be raised (orlowered), or an effect used for a voice changer may be applied toprocess the re-listening sound to be heard differently from thereal-time sound by sound pitch conversion technology. In addition, voicequality of the re-listening sound can be converted by replacing aspectrum envelope, separation of pitch information, and a spectrumenvelope, or sound pitch of the re-listening sound can be changed byreplacing pitch information, according to the Vocoder technology that isone of so-called sound and synthesis technologies.

1.4. Conclusion

The configuration of the hearing-aid system according to the embodimentand the re-listening reproduction processing thereof have been describedabove. According to the embodiment, a user can immediately reproducemissing re-listening sound during conversation even if the user missesthe sound, and can smoothly switch between reproduction of re-listeningsound and ordinary reproduction (that is, listening of the real-timesound). In addition, the hearing-aid system according to the embodimentcan provide re-listening sound that can be easily heard.

2. Second Embodiment

Next, a hearing-aid apparatus 300 according to a second embodiment ofthe present disclosure will be described on the basis of FIGS. 10 and11. The hearing-aid apparatus 300 according to the embodiment has aconfiguration in which the hearing-aid system according to the firstembodiment is organized in one device. Hereinafter, the minimumconfiguration of the hearing-aid apparatus 300 according to theembodiment and a configuration example in a case in which functions suchas a noise cancellation function are added to the minimum configurationwill be described. In addition, FIG. 10 corresponds to the configurationof the hearing-aid system according to the first embodiment illustratedin FIG. 1, and FIG. 11 corresponds to the configuration of thehearing-aid system according to the first embodiment illustrated in FIG.2. Therefore, detailed description of the respective functions will beomitted.

(1) Minimum Configuration

First, a minimum functional configuration of the hearing-aid apparatus300 according to the embodiment will be described on the basis of FIG.10. The hearing-aid system illustrated in FIG. 10 is enabled to performthe re-listening reproduction function in addition to an ordinaryhearing-aid function as minimum functions.

The hearing-aid apparatus 300 includes a sound input unit 310, a chasingreproduction signal processing unit 320, a signal synthesis selectionunit 330, a control input unit 340, a sound output unit 350, a firstsound storage unit 361, and a second sound storage unit 363 asillustrated in FIG. 10. In this hearing-aid apparatus 300, thefunctional unit that transmits and receives data between devices isomitted as compared with the hearing-aid system in FIG. 1. The soundinput unit 310 corresponds to the sound input unit 110, and the chasingreproduction signal processing unit 320 corresponds to the chasingreproduction signal processing unit 213. In addition, the signalsynthesis selection unit 330 corresponds to the signal synthesisselection unit 140, and the control input unit 340 corresponds to thecontrol input unit 220. In addition, the sound output unit 350corresponds to the sound output unit 150, the first sound storage unit361 corresponds to the first sound storage unit 231, and the secondsound storage unit 233 corresponds to the second sound storage unit 363.Since the functions of the respective functional units are similar tothose in the first embodiment, detailed description will be omitted. Onedevice that includes the re-listening reproduction function can beconfigured as described above.

(2) Applied Configuration

Next, a configuration example of a hearing-aid apparatus 300 that isachieved by further adding functions to the hearing-aid system with theminimum configuration illustrated in FIG. 10 as a base will be describedon the basis of FIG. 11. The hearing-aid apparatus 300 illustrated inFIG. 11 includes a sound analysis function, a noise cancellationfunction, and the like in addition to the hearing-aid function and there-listening reproduction function that are minimum functions.Hereinafter, the functions newly added to the hearing-aid system withthe first-time configuration illustrated in FIG. 10 will be mainlydescribed in comparison with the hearing-aid system according to thefirst embodiment illustrated in FIG. 2.

The functions of the hearing-aid apparatus 300 can be roughly classifiedinto a sound input and output unit 302 and a re-listening reproductionprocessing unit 304 as illustrated in FIG. 11. The sound input andoutput unit 302 is a processing unit that implements the function of theaudio device 100 in the hearing-aid system in FIG. 2, and there-listening reproduction processing unit 304 is a processing unit thatimplements the function of the information processing device 200 in thehearing-aid system in FIG. 2. The functional unit of transmitting andreceiving data between devices is omitted from such a hearing-aidapparatus 300 as compared with the hearing-aid system in FIG. 2, in thesimilar manner to that in the hearing-aid apparatus 300 in FIG. 10.

That is, the sound input and output unit 302 includes a sound input unit310, a first signal synthesis selection unit 331, a real-time soundprocessing unit 333, a second signal synthesis selection unit 335, and acancelation signal generation unit 370. These components correspond tothe sound input unit 110, the first signal synthesis selection unit 141,the real-time sound processing unit 143, the second signal synthesisselection unit 145, the sound output unit 150, and the cancelationsignal generation unit 160 in the audio device 100 in FIG. 2,respectively.

In addition, the re-listening reproduction processing unit 304 includesa chasing reproduction signal processing unit 320 and a sound analysisprocessing unit 321. These correspond to the chasing reproduction signalprocessing unit 213 and the sound analysis processing unit 212 in theinformation processing device 200 in FIG. 2. In addition, thehearing-aid apparatus 300 includes a first sound storage unit 361, asecond sound storage unit 363, and an information presentation unit 380.These correspond to the first sound storage unit 231, the second soundstorage unit 233, and the information presentation unit 240 in thehearing-aid system in FIG. 2, respectively. Since the functions of therespective functional units are similar to those in the firstembodiment, detailed description will be omitted. However, it ispossible to configure a device that has the functions of the hearing-aidsystem in FIG. 2 as described above.

3. Hardware Configuration Examples

At last, hardware configuration examples of the audio device 100, theinformation processing device 200, and the hearing-aid apparatus 300according to the embodiment will be described. Since these devices canbe configured in a similar manner, description will be given below byexemplifying the information processing device 200. FIG. 12 is ahardware configuration diagram illustrating the hardware configurationof the information processing device 200 according to the embodiment.

The information processing device 200 according to the presentembodiment can be implemented as a processing device including acomputer, as described above. As illustrated in FIG. 12, the informationprocessing device 200 is configured to include a central processing unit(CPU) 901, a read only memory (ROM) 902, a random access memory (RAM)903, and a host bus 904 a. In addition, the information processingdevice 200 is configured to include a bridge 904, an external bus 904 b,an interface 905, an input device 906, an output device 907, a storagedevice 908, a drive 909, a connection port 911, and a communicationdevice 913.

The CPU 901 functions as an arithmetic processing device and a controldevice and controls the overall operation in the information processingdevice 200 according to various programs. Further, the CPU 901 may be amicroprocessor. The ROM 902 stores programs, operation parameters andthe like used by the CPU 901. The RAM 903 temporarily stores programsused in execution of the CPU 901, parameters appropriately changed inthe execution, and the like. These components are interconnected via thehost bus 904 a formed by a CPU bus or the like.

The host bus 904 a is connected to the external bus 904 b such asperipheral component interconnect/interface (PCI) bus through the bridge904. Moreover, the host bus 904 a, the bridge 904, and the external bus904 b are not necessarily configured as separate components, and thefunctions of them may be incorporated into a single bus.

The input device 906 is configured to include input means through whicha user can input information, such as a mouse, a keyboard, a touchpanel, a button, a microphone, a switch, a lever, an input controlcircuit that generates an input signal on the basis of the input by theuser and outputs it to the CPU 901, and the like. The output device 907includes, in one example, a display device such as a liquid crystaldisplay (LCD) device, an organic light emitting diode (OLED) device, ora lamp, and a sound output device such as a speaker.

The storage device 908 is an example of the storage unit of theinformation processing device 200 and is a device for storing data. Thestorage device 908 may include a recording medium, a recording devicethat records data in the recording medium, a readout device that readsout data from the recording medium, a deletion device that deletes datarecorded in the recording medium and the like. The storage device 908drives a hard disk, and stores a program executed by the CPU 901 andvarious kinds of data.

The drive 909 is a reader-writer for a recording medium, and is built inthe information processing device 200 or is externally attached thereto.The drive 909 reads out information recorded in a mounted magnetic disk,optical disk, magneto-optical disc, or removable recording medium suchas a semiconductor memory, and outputs the information to the RAM 903.

The connection port 911 is an interface connected to an external deviceand is a port for connecting an external device that is capable oftransmitting data through, in one example, a universal serial bus (USB).Furthermore, the communication device 913 is, in one example, acommunication interface formed by a communication device or the like forconnecting to a communication network 5. Furthermore, the communicationdevice 913 may be a communication device compatible with a wirelesslocal area network (LAN), a communication device compatible with awireless USB, or a wired communication device that communicates withwire.

The preferred embodiment(s) of the present disclosure has/have beendescribed above with reference to the accompanying drawings, whilst thepresent disclosure is not limited to the above examples. A personskilled in the art may find various alterations and modifications withinthe scope of the appended claims, and it should be understood that theywill naturally come under the technical scope of the present disclosure.

Further, the effects described in this specification are merelyillustrative or exemplified effects, and are not limitative. That is,with or in the place of the above effects, the technology according tothe present disclosure may achieve other effects that are clear to thoseskilled in the art from the description of this specification.

Additionally, the present technology may also be configured as below.

(1)

An information processing apparatus including:

a reproduction processing unit that performs reproduction of a recordedsound on a basis of a reproduction start instruction for startingre-listening of the recorded sound from a position tracking back apredetermined time from a reproduction start time, at which thereproduction start instruction is input, to a position of a presenttime.

(2)

The information processing apparatus according to (1),

in which the reproduction processing unit reproduces the recorded soundat a first reproduction speed to a position of the reproduction starttime and reproduces the recorded sound at a second reproduction speedfrom the position of the reproduction start time to the position of thepresent time.

(3)

The information processing apparatus according to (2),

in which, when a reproduction completion instruction of the recordedsound is input before a position of reproduction of the recorded soundreaches the reproduction start time, the reproduction processing unitmoves the position of reproduction to the reproduction start time.

(4)

The information processing apparatus according to (2) or (3),

in which the second reproduction speed is equal to or faster than thefirst reproduction speed.

(5)

The information processing apparatus according to any one of (2) to (4),

in which the second reproduction speed is faster than an unmultipliedspeed.

(6)

The information processing apparatus according to any one of (1) to (5),

in which the reproduction processing unit performs the reproduction byskipping at least a part of a section from the reproduction start timeto the present time.

(7)

The information processing apparatus according to (6),

in which a target section to be skipped is a time section in which nohuman voice is present or a no-sound section.

(8)

The information processing apparatus according to any one of (1) to (7),further including:

a cancelation signal generation unit that executes a noise cancellationfunction at least during reproduction of the recorded sound.

(9)

The information processing apparatus according to any one of (1) to (8),including:

a sound analysis processing unit that analyzes the recorded sound,

in which the reproduction processing unit performs sound emphasisprocessing on a basis of an analysis result obtained by the soundanalysis processing unit.

(10)

The information processing apparatus according to any one of (1) to (8),including:

a sound analysis processing unit that analyzes the recorded sound; and

an information presentation unit that presents information related tothe sound on a basis of an analysis result obtained by the soundanalysis processing unit.

(11)

The information processing apparatus according to (10),

in which the information presentation unit that causes a display todisplay a sound waveform of the acquired sound.

(12)

The information processing apparatus according to any one of (1) to(11), including:

a real-time sound processing unit that performs predetermined soundcorrection processing on input sound and outputs real-time sound.

(13)

The information processing apparatus according to (12), furtherincluding:

a signal synthesis selection unit that performs reproduction processingof the recorded sound and the real-time sound on a basis of a controlinput.

(14)

The information processing apparatus according to (13),

in which the signal synthesis selection unit does not reproduce thereal-time sound at a time of reproduction of the recorded sound.

(15)

An information processing system including:

a first device that acquires and outputs sound; and

a second device that includes a reproduction processing unit thatperforms reproduction of a recorded sound acquired by the first deviceon a basis of a reproduction start instruction for starting re-listeningof the recorded sound from a position tracking back a predetermined timefrom a reproduction start time, at which the reproduction startinstruction is input, to a position of a present time.

(16)

The information processing system according to (15),

in which the first device further includes

-   -   a data transmission unit that compresses data to be transmitted        to the second device and transmits the compressed data, and    -   a data reception unit that expands data received from the second        device.        (17)

The information processing system according to (16),

in which the first device further includes a sound feature calculationunit that calculates a feature amount included in waveform data of theacquired sound, and

the data transmission unit transmits the feature amount to the seconddevice.

(18)

The information processing system according to (16),

in which the first device further includes a sound feature calculationunit that calculates a feature amount included in waveform data of theacquired sound, and

the data transmission unit determines, on a basis of the feature amount,a sound section to be transmitted to the second device in the acquiredsound and transmits the sound section to the second device.

(19)

The information processing system according to any one of (15) to (18),

in which the second device includes an information presentation unitthat presents information related to reproduction of recorded sound.

(20)

A program for causing a computer to function as:

a reproduction processing unit that performs reproduction of a recordedsound on a basis of a reproduction start instruction for startingre-listening of the recorded sound from a position tracking back apredetermined time from a reproduction start time, at which areproduction start instruction is input, to a position of a presenttime.

REFERENCE SIGNS LIST

-   100 audio device-   110 sound input unit-   120 audio device data transmission unit-   121 sound feature calculation unit-   123 time stamp information generation unit-   130 audio device data reception unit-   140 signal synthesis selection unit-   141 first signal synthesis selection unit-   143 real-time sound processing unit-   145 sound signal synthesis selection unit-   150 sound output unit-   160 cancelation signal generation unit-   200 information processing device-   210 reproduction processing unit-   211 information processing device data reception unit-   212 sound analysis processing unit-   213 reproduction signal processing unit-   215 information processing device data transmission unit-   220 control input unit-   231 first sound storage unit-   233 second sound storage unit-   240 information presentation unit-   300 hearing-aid apparatus-   302 sound input and output unit-   304 reproduction processing unit-   310 sound input unit-   320 reproduction signal processing unit-   321 sound analysis processing unit-   330 signal synthesis selection unit-   331 first signal synthesis selection unit-   333 real-time sound processing unit-   335 sound signal synthesis selection unit-   340 control input unit-   350 sound output unit-   361 first sound storage unit-   363 second sound storage unit-   370 cancelation signal generation unit-   380 information presentation unit

The invention claimed is:
 1. A hearing-aid apparatus comprising: areproduction processor configured to process a recorded audio signalbased on a relistening instruction inputted at an instruction time; aninput receiver configured to receive user input; and a sound synthesizerconfigured to generate a sound signal to be outputted, the sound signalbeing generated based on one of: a relistening signal from thereproduction processor and the recorded audio signal, wherein therelistening instruction causes the reproduction processor to process therecorded audio signal such that the relistening signal includes a firstportion of the recorded audio signal starting from a start time set at apredetermined amount of time before the instruction time, and alsoincludes a second portion corresponding to a catchup segment of therecorded audio signal from the instruction time to a current time,wherein the predetermined amount of time is based on input by a user andcorresponds to an amount of time to track back from the instructiontime, wherein, the sound synthesizer generates the sound signal suchthat: the second portion has a faster speed than a speed of a remainderof the sound signal, and upon reaching a current time of the recordedaudio signal, outputs the sound signal at a real-time speed, and whereinthe input receiver is configured to receive user input of any one or anycombination of: the relistening instruction, a selection of thepredetermined amount of time for the first portion, and a stopinstruction to cause the sound synthesizer to stop generating the soundsignal based on the relistening signal before an end of the relisteningsignal is reached.
 2. The hearing-aid apparatus of claim 1, furthercomprising: a noise processor configured to produce a noise-cancellationsignal for cancelling a noise in the recorded audio signal.
 3. Thehearing-aid apparatus according to claim 1, wherein wherein, when thestop instruction is received before the end of the relistening signal isreached, the sound synthesizer commences generation of the sound signalfrom the current time of the recorded audio signal.
 4. The hearing-aidapparatus according to claim 1, further comprising: a real-time soundprocessor configured to perform a predetermined sound correction processon an input audio signal and to output a real-time corrected signal tothe sound synthesizer and the reproduction processor as the recordedaudio signal.
 5. The hearing-aid apparatus according to claim 1, furthercomprising: a controller configured to control the signal synthesizer togenerate the sound signal based on the relistening signal from thereproduction processor or the recorded audio signal.
 6. A hearing-aidapparatus comprising: an audio device configured to acquire an audiosignal, produce a digitized signal from the audio signal, produce arecorded signal from the digitized signal, and output a sound signal;and an information processor that includes: a user interface configuredto receive input from a user, and a reproduction processor configured toprocess the recorded audio signal based on a relistening instructioninputted at an instruction time, wherein the audio device includes asound synthesizer configured to generate the sound signal based on oneof: a relistening signal from the reproduction processor and therecorded audio signal, wherein the relistening instruction causes thereproduction processor to process the recorded audio signal such thatthe relistening signal includes a first portion of the recorded audiosignal starting from a start time set at a predetermined amount of timebefore the instruction time, and also includes a second portioncorresponding to a catchup segment of the recorded audio signal from theinstruction time to a current time, wherein the predetermined amount oftime is based on input by a user and corresponds to an amount of time totrack back from the instruction time, wherein, the sound synthesizergenerates the sound signal such that: the second portion has a fasterspeed than a speed of a remainder of the sound signal, and upon reachinga current time of the recorded audio signal, outputs the sound signal ata real-time speed, and wherein the user interface is configured toreceive user input of any one or any combination of: the relisteninginstruction, a selection of the predetermined amount of time for thefirst portion, and a stop instruction to cause the sound synthesizer tostop generating the sound signal based on the relistening signal beforean end of the relistening signal is reached.
 7. The hearing-aidapparatus of claim 6, wherein the audio device includes a noiseprocessor configured to produce a noise-cancellation signal forcancelling a noise in the recorded audio signal.
 8. The hearing-aidapparatus according to claim 6, wherein wherein, when the stopinstruction is received before the end of the relistening signal isreached, the sound synthesizer commences generation of the sound signalfrom the current time of the recorded audio signal.
 9. The hearing-aidapparatus according to claim 6, wherein the audio device includes areal-time sound processor configured to perform a predetermined soundcorrection process on the digitized signal and to output a real-timecorrected signal to the sound synthesizer and the reproduction processoras the recorded audio signal.
 10. The hearing-aid apparatus according toclaim 6, wherein the information processor includes a controllerconfigured to control the signal synthesizer to generate the soundsignal based on the relistening signal from the reproduction processoror the recorded audio signal.
 11. The hearing-aid apparatus according toclaim 6, wherein the information processor is a processor included in asmartphone or an information processing terminal configured tocommunicate with the audio device.
 12. The hearing-aid apparatusaccording to claim 6, wherein the audio device includes an earphone or aheadphone.
 13. A non-transitory computer-readable medium comprisingcomputer-executable instructions that, when executed by a processor of acomputer, cause the computer to function as a hearing-aid apparatus,wherein the hearing-aid apparatus comprises: a reproduction processorconfigured to process a recorded audio signal based on a relisteninginstruction inputted at an instruction time, an input receiverconfigured to receive user input, and a sound synthesizer configured togenerate a sound signal to be outputted, the sound signal beinggenerated based on one of: a relistening signal from the reproductionprocessor and the recorded audio signal, wherein the relisteninginstruction causes the reproduction processor to process the recordedaudio signal such that the relistening signal includes a first portionof the recorded audio signal starting from a start time set at apredetermined amount of time before the instruction time, and alsoincludes a second portion corresponding to a catchup segment of therecorded audio signal from the instruction time to a current time,wherein the predetermined amount of time is based on input by a user andcorresponds to an amount of time to track back from the instructiontime, wherein the relistening signal is configured to cause the soundsynthesizer to generate the sound signal such that: the second portionhas a faster speed than a speed of a remainder of the sound signal, andupon reaching a current time of the recorded audio signal, the soundsignal is output at a real-time speed, and wherein the input receiver isconfigured to receive user input of any one or any combination of: therelistening instruction, a selection of the predetermined amount of timefor the first portion, and a stop instruction to cause the soundsynthesizer to stop generating the sound signal based on the relisteningsignal before an end of the relistening signal is reached.
 14. Thenon-transitory computer-readable medium according to claim 13, whereinthe hearing-aid apparatus comprises a noise processor configured toproduce a noise-cancellation signal for cancelling a noise in therecorded audio signal.
 15. The non-transitory computer-readable mediumaccording to claim 13, wherein the hearing-aid apparatus comprises areal-time sound processor configured to perform a predetermined soundcorrection process on an input audio signal and to output a real-timecorrected signal as the recorded audio signal.
 16. The non-transitorycomputer-readable medium according to claim 13, wherein the hearing-aidapparatus comprises a controller configured to control the signalsynthesizer to generate the sound signal based on the relistening signalfrom the reproduction processor or the recorded audio signal.